Three dimensional image processing

ABSTRACT

A method of preparing on a first device object files used for rendering two-dimensional images of threes-dimensional objects on a second device by processing object files containing data defining triangle primitives representing at least one three-dimensional object to determine a rendering order in which the triangle primitives may be rendered for creating a two-dimensional image in which the hidden surfaces are removed. A plurality of planar triangle primitives corresponding to a plurality of surface portions of the at least one object as vertex data are stored on the first device. The method can be used in a mobile phone.

This application is the National Stage of International Application No.PCT/EP2002/011507, International Filing Date, Oct. 15, 2002, whichdesignated the United States of America, and which internationalapplication was published under PCT Article 21(2) as WO Publication No.WO 2004/036504 A1.

The present invention relates to a method of preparing object files on afirst device that are used for rendering two-dimensional images ofthree-dimensional objects on a second device. The object files containdata defining polygon primitives representing at least onethree-dimensional object. The data in the object file is rearranged fordetermining a rendering order in which said polygon primitives may berendered in which the hidden surface problem is solved. The inventionrelates further to a mobile communication terminal comprising imageprocessing means for rendering images of three-dimensional objects on adisplay. The present invention relates also to a method of renderingtwo-dimensional images of three-dimensional objects by processing datadefining polygon primitives with a reduced amount of vertex data. Theinvention further relates to a mobile communication terminal comprisingimage processing means for rendering images of three-dimensional objectson a display capable of displaying an image on the basis of a reducedamount of vertex data.

BACKGROUND ART

Until recently, animated three-dimensional graphics were exclusivelyapplied in devices with a relatively high processing power such as, forinstance, in generating displays of the appearance of an object fromseveral angles on a workstation or a PC during computer aided design andcomputer aided manufacture, in electronic video arcade games, or indriving- or flight simulators.

Three-dimensional graphics now also find their way into devices with arelatively low processing power such as mobile phones, personal digitalassistants (PDA) and hand-held video games.

To provide animation, it is necessary to update the image at ratesapproaching 20 to 30 frames per second (lower rates are acceptable forsome applications), so that an observer perceives the image as acontinuous change rather than a chain of static images.

The speed at which the apparatus can perform calculations, coupled withthe minimum acceptable frame rate, provides a limit to the number ofcalculations which can be performed to generate each image.

In three-dimensional animation, it is necessary to calculate for eachimage the projection of the three-dimensional objects on two dimensions.For complex images a large amount of calculations can be required.

Image synthesis is usually performed by representing an object as anumber of polygon primitives defined by vertices, projecting the polygonprimitives onto an image plane, and then processing each polygonprimitive in order to define, for each pixel of the projected polygon,the pixel color.

A major problem in rendering three-dimensional graphics is the removalof hidden surfaces. From the viewpoint, the nearer surfaces shouldobscure the further parts.

A common approach is to test, when a given pixel is rendered, whether itlies in front (and therefore occludes) or behind, (and therefore isoccluded by) the pixel (if any) which is already within the imagebuffer. Accordingly, each pixel needs to be assigned a depth value. Thistechnique is referred to as the “z-buffer algorithm”. This technique isa technically simple approach, but it is resource intensive in terms ofstorage and processing power since it involves a large number ofcomparison operations (on the order of the number of pixels in the imagebuffer), for real time operation.

The “Painter's Algorithm” examines the coordinates of the vertices ofthe polygons, and ranks the polygons into order of decreasing distancefrom the viewpoint. The polygons are then rendered in order ofdecreasing distance, so that the pixels of the most distant polygons arefirst written into the image buffer, and are overwritten as necessary bythose of nearer polygons. However, this technique requires the sortingoperation of all polygons to be repeated each time the viewpointchanges, and consequently, requires a large amount of computation inreal time.

In a known technique, polygons are viewed as having two faces; one facedirected inwardly towards the interior of the object (the “back” face)and one face directed outwardly of the object (the “front” face). For asolid object, the “back” face of a polygon is never visible; onlypolygons which face forwards therefore need to be displayed.Accordingly, a decision is made prior to rendering whether, from theviewing angle concerned, a given polygon is showing its front (outer) orback (inner) face, and only those polygons which are facing forward arerendered. This method of back face culling reduces the problem of hiddensurface removal, but does not eliminate the problem altogether.

A further known technique is known as “binary space partitioning” (BSP).In this technique, prior to animation, a pre-processing stage isperformed in which a tree-structured model is built in memory, each nodein the tree being a polygon on a space partitioning plane and pointingto successor and predecessor nodes (if any) one of the nodes contains atree consisting of polygons strictly on one side of the partitioningplane, while the other node contains the rest. One can then recursivelyprocess the tree (descending first to the node further away from thecamera) and render polygons in back to front order.

Another known technique is polygon stripping i.e. to convert independenttriangles or polygons into triangle strips. In this technique, a firsttriangle is identified with an either clockwise or counterclockwisewinding, and then all subsequent triangles in the strip will alternatewinding. The strip is continued until the triangle strip runs off theedge of the object or runs into a polygon that is already a part ofanother strip.

Traditionally, due to their extensive consumption of system resources,three-dimensional graphics engines belong to high performance systemssuch as workstations with powerful processors and large memories. Therestrictions posed by limited calculation power are more evident ondevices which cannot be provided with the fastest processors. Examplesof such devices are mobile phones, personal digital assistants (PDA) andhandheld video games. There is therefore a need to provide effectivealgorithms so that three-dimensional real time animated graphics can behandled by the processing power available in the above-mentioneddevices.

DISCLOSURE OF THE INVENTION

On this background, it is an object of the present invention to providea method of preparing on a first device object files used for renderingtwo-dimensional images of three-dimensional objects on a second deviceto determine a rendering order in which said triangle primitives may berendered for creating a two-dimensional image in which the hiddensurfaces are removed, which allows real time three-dimensional imagerendering on devices with relatively low processing power.

This object is achieved in accordance with claim 1 by providing a methodof preparing object files on a first device that are used for renderingtwo-dimensional images of three-dimensional objects on a second deviceby processing object files containing data defining triangle primitivesrepresenting at least one three-dimensional object to determine arendering order in which said triangle primitives may be rendered forcreating a two-dimensional image in which the hidden surfaces areremoved comprising the steps of:

a) storing a plurality of planar triangle primitives corresponding to aplurality of surface portions of said at least one object in an objectfile, wherein said plurality of triangle primitives are represented byvertex data arranged according to an original order,

b) selecting a triangle that does not potentially occlude any of theunprocessed triangles to start a triangle strip,

c) selecting a neighboring triangle to add to the last triangle in thetriangle strip,

d) determining whether the selected triangle can potentially occlude anyof the unprocessed triangles,

e) adding the selected triangle to the strip and going back to step c)if the selected triangle cannot potentially occlude any of theunprocessed triangles,

f) going back to step b) to start a new triangle strip if the selectedtriangle can potentially occlude any of the unprocessed triangles, and

g) repeating steps b) to f) until all the triangles in the object fileare processed.

Thus, the object files can be reorganized in triangle strips that can bedisplayed in the order in which they were created whilst the hiddensurface problem is solved, i.e. the device that receives the thusprepared object files can simply display the triangle strips and thetriangles in the order in which they are stored in the object filewithout the need for calculations to solve the hidden surface problem.The number of calculations to be made by the image-processing device isthus significantly reduced by pre-sorting the vertex data such that therendering order of the triangle primitives in the triangle strips iscorrect, i.e. the hidden surfaces are correctly removed, independent ofthe viewing direction and the camera position. Thus, the second deviceonly needs to calculate the rendering order of the objects and then, foreach frame of animation, to render the front facing triangle primitivesof the objects in the order that they are stored in the object files.Compared to Z-buffering, for example, a large reduction in the amount ofcalculations that are required can be obtained as no comparison of thepixel's previous Z-value is required.

Step b) of the method preferably comprises a heuristic search toidentify those triangles for starting a strip that potentially allowlong strips to be formed, preferably by selecting triangles that do nothave many unprocessed neighbors.

For some objects there is no order in which the triangle primitives canbe placed in which the hidden surfaces are removed correctly for allviewing directions because of conflicting occlusion priorities. Thus, itis necessary to split such an object into two objects and presort thetriangle primitives for each thus created object.

The method preferably comprises on said first device further the stepof:

-   -   dividing the object file into two if at step b) it is impossible        to find a triangle not occluding other, still unprocessed,        triangles.

The object is preferably divided by using a partitioning plane, withwhich at least one of the unprocessed triangles remains on a differentside than the unprocessed triangles it can potentially occlude.

Preferably, a plane of an arbitrary unprocessed triangle is used and thetriangle is associated on the outward side of the splitting plane, thusleaving all the polygons that it may cover on the other side.

While dividing the polygons into two sets, one may have to splitpolygons that intersect the partitioning plan. In general, it isbeneficial to split polygons into two sets such that the number ofpolygons in the two sets is as equal as possible.

The method preferably comprises further the step of determining thefront side and the triangle planes of the triangles in the object filewherein the determination if a selected triangle can occlude any of theunprocessed triangles comprises the steps of determining whether:

g) the selected triangle has one or more vertices on the front side ofthe plane of any of the unprocessed triangles, and

h) whether any of the unprocessed triangles positively identified instep g) have one or more vertices on the backside of the plane of theselected triangle.

The method may further comprise on said first device the following stepsbefore step b)

i) determining if the object has symmetry in the XY-plane of the objectspace,

j) storing a determined plane of symmetry, and

k) removing all vertices in the object file from one side of adetermined plane of symmetry, and

repeating steps i) to k) for the XZ-plane of the object space and forthe YZ-plane of the object space.

The method preferably comprises the step of:

l) transferring said object file or files from said first device to asecond device after step f) or k) and storing the object files on saidsecond device.

The first device preferably comprises a higher performance imageprocessing means, such as used in a PC or a workstation and said seconddevice preferably comprises a lower performance processing means such asused in a mobile communication terminal or a PDA.

Thus, the calculation intensive part of the image rendering process canbe carried out in advance on a device with a high processing power suchas a workstation or a PC, and the animated three-dimensional graphicscan be rendered in real time on a device with relatively low processingpower such as a mobile phone, a PDA or a handheld video game.

It is another object of the present invention to provide a method fordisplaying object files obtained by the method above on the seconddevice. This object is achieved in accordance with claims 9, byproviding such a method, comprising the steps of:

m) determining the viewing direction, and

n) determining the object order, preferably using the painter'salgorithm and displaying said object or objects on said second device.

Preferably, the method further comprises the step of:

o) culling all triangles facing backwards relative to said viewingdirection.

The method advantageously comprises the steps of:

p) retrieving planes of symmetry stored in the object file, and

q) creating a duplication of the vertex data present in the object file,said duplication of the vertex data having the same order as theoriginal vertex data and being stored in the object file after theoriginal vertex data, and flipping the sign of the vertex values on theaxis orthogonal to the respective retrieved plane of symmetry of theduplicated vertices.

Recreate vertices by flipping the sign of one of the coordinates, canonly be carried out if and only if the plane of symmetry is one of x=0,y=0, z=0. If the plane is, e.g., x=1, and there is a vertex 0,1,2, themirror pair cannot be obtained by flipping the sign of x component.Instead it would be necessary to mirroring it with respect to thecoordinate x=1, so that the mirror pair would be 2,1,2. The method maytherefore comprise the steps of translating and rotating the object sothat the axes of symmetry match with x=0, y=0, or z=0. During therendering operation the inverse steps are applied to the whole object.

The method may further comprise the steps of:

r) displaying the triangle strips in the same order as they werecreated, and

s) displaying the triangles in the triangle strips in the same order asthey were added to the strip.

Thus, the triangles can be displayed directly as they are stored in theobject file, independent of the viewing direction and camera position,without the second device needing to carry out calculations to resolvethe hidden surface problem.

Preferably, the second device comprises a lower performance processingmeans such as used in a mobile communication terminal or a PDA, and mostpreferably the second device is a mobile phone.

The image rendering process is preferably carried out by the main CPU ofthe device, and all the further image rendering means is softwareembedded.

It is another object of the present invention to provide a mobilecommunication terminal capable of displaying real-time rendered animatedthree-dimensional graphics. This object is achieved in accordance withclaim 16 by providing such a mobile communication terminal thatcomprises:

-   -   means to store object files containing triangle strips, and    -   means to display said triangle strips in the order in which they        have been created, which preferably corresponds to the order in        which triangle strips are stored in the object files.

The triangles in the triangle strips can be displayed in the same orderas they were added to the strip, which preferably corresponds to theorder in which they are stored in the triangle strips.

Preferably, a rendering order of the objects is determined using abinary space partitioning method. The rendering order of the objectfiles is sorted based on the current viewing direction and position sofar away objects are rendered first.

The mobile communication terminal may comprise means for culling alltriangles facing backwards relative to the viewing direction.

The mobile communication terminal may further comprise

-   -   means for retrieving planes of symmetry stored in the object        file, and    -   means for duplicating all vertex data present in the object file        and flipping the sign of the vertex values on the axis        orthogonal to the respective plane of symmetry for the newly        created triangles.

The image rendering process is preferably carried out by the main CPU ofthe device, and all further image rendering means is software embedded.

It is yet another object of the present invention to provide a method ofpreparing on a first device object files used for renderingtwo-dimensional images of three-dimensional objects on a second deviceby processing data defining triangle primitives representing at leastone three-dimensional object. The method requires a reduced amount ofcalculations and therefore allows real time three-dimensional imagerendering on devices with relatively low processing power. This objectis achieved in accordance with claim 22 by providing such a method ofsuch kind comprising on said first device the steps of:

a) storing a plurality of planar triangle primitives corresponding to aplurality of surface portions of said at least one object, wherein saidplurality of triangle primitives are represented by vertex data,

b) determining the planes of symmetry of said three-dimensional object,

c) storing planes of symmetry data in the object file,

d) culling all vertex data from one side of each said planes ofsymmetry,

e) transferring said object file from said first device to said seconddevice, and

f) displaying said image on said second device by mirroring said vertexdata about said planes of symmetry and rendering said image.

Preferably, planes of symmetry that are not perpendicular with one ofthe X-, Y- or Z-axis of the object coordinate system are ignored.Alternatively, the object may be rotated and translated so that the axesof symmetry align with one or more of the planes X=0, Y=0, or Z=0.

The first device may comprise higher performance image processing means,such as used in a PC or a workstation and said second device maycomprise lower performance processing means such as used in a mobilecommunication terminal, a PDA or a handheld video game.

The second terminal is preferably a mobile phone.

It is a further object of the invention to provide a mobilecommunication terminal capable of displaying two-dimensional images ofthree-dimensional objects with a reduced amount of calculations. Thisobject is achieved in accordance with claim 26 by providing a mobilecommunication terminal comprising image processing means for renderingimages of three-dimensional objects on a display, means for storingvertex data, means for storing planes of symmetry data and means formirroring vertex data in a symmetry plane.

Further objects, features, advantages and properties of the methods andmobile communication terminals according to the invention will becomeapparent from the detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following detailed portion of the present description, theinvention will be explained in more detail with reference to theexemplary embodiments shown in the drawings, in which

FIG. 1 is a diagram illustrating a front face of a triangle,

FIG. 2 is a flow chart showing two stages of creating object files ofthree-dimensional objects on a first device,

FIG. 3 is a detailed flow chart showing the steps for creating objectsfiles with reduced vertex data by removing mirrored vertices,

FIG. 4 is a detailed flow chart showing the steps for creating trianglestrips on the first device,

FIG. 5 is a detailed flowchart illustrating the process to determine ifa selected triangle can potentially occlude an unprocessed triangle,

FIG. 6 is a flow chart showing the steps for displaying thethree-dimensional objects as a two-dimensional image on a second device,

FIG. 7 is a detailed flow chart showing the steps for restoring thetriangles removed in the steps shown in FIG. 3, and

FIG. 8 is a block diagram of a mobile phone.

DETAILED DESCRIPTION

Three-dimensional graphics systems provide a virtual three-dimensional(3-D) environment in which objects, light sources and a viewing positionmay be manipulated within a notional three-dimensional space. Theviewing position provides a window into the virtual three-dimensionalworld, often referred to as “world space”. This world space itself onlyexists as data defining three-dimensional coordinates and othervariables, such as color and reflectivity. The two-dimensional imagepresented at the notional window must be created by processing thethree-dimensional image data, in a procedure known as rendering.

Systems are known in which objects in the three-dimensional space arerepresented by the coordinate locations of selected points on thesurface of the object.

However, rendering consists of allocating picture element values totwo-dimensional regions of the image. Therefore, these coordinate pointsare converted into notional regions by effectively connecting the pointsby straight lines to provide a three-dimensional net of flat,two-dimensional triangles, defined by a number of vertices. The presentinvention will be described by using triangles, but the reader skilledin the art will appreciate that the techniques described also apply toother flat polygons.

The picture element values are generated on a triangle-by-trianglebasis, until the whole object has been tessellated, i.e. broken downinto triangles. The vertex data thus created is stored in an originalorder in the object file.

The triangles are saved in a manner that defines a normal vectoridentifying the front face (solid objects always have a front face and aback face, the latter never being visible) of the triangles. The “front”face is defined by vector normal to the surface of the triangle, withthe direction of the vector (and hence the direction of the front face)being determined by a right-hand (or, optionally, left-hand) ruletraversing the three vertices of the triangle in an order determined bythe manner in which they are stored in the triangle's data structure.For instance, in FIG. 1, the three points 22,23,24 determine thedirection of the normal vector 25, and hence the side of the triangle 21that is to be defined as the front face 26. The normal vector 25 is e.g.determined by the cross product of two vectors along the edges of thetriangle. The above steps are all carried out on a workstation, PC orsimilar high performance computer.

The picture element values generated by the rendering process are savedon the workstation. In order to maintain the illusion of a solidthree-dimensional object when displaying the triangles on a display, itis essential that the triangles are sorted in an order which ensuresthat occluded triangles are rendered before the triangles which occludethem. Real-time sorting of the triangle order is unfeasible on e.g. amobile phone with relatively low performance hardware for processing thedata. In the prior art techniques, this problem was solved partially bypreparing a predetermined rendering order, which could, however, changeas changes are made to the viewing direction or the location of theviewpoint. As will be shown here, the technique according to theinvention overcomes this problem.

The term “object” as used here includes single objects and also acollection of several separate objects, as long as the objects in thatcollection do not move with respect to each other during the animation,e.g. a table and a flower pot and a chair next to the table can beconsidered as one object, if they don't move with respect to each other.If in the course of animation the chair moves, it can't be the sameobject as the desk.

The stages for providing a three-dimensional graphics environment on aworkstation or other powerful computer (not shown) are shown in FIG. 2.Initially, data defining the objects is prepared. Thus, at step 35, afirst object is modeled. Similarly, at step 40 a second object ismodeled and so on until at step 45 an Nth object is modeled. Each objectis modeled with reference to its own set of coordinates, often referredto as its own modeling space. Thus, each set of object data defines ashape, and the data required to reproduce said shape in a graphicsenvironment is stored in an object file in the form of vertex data.

The next stage consists of steps 50, 52, 55 and 60, which form part ofthe pre-processing stage. At step 50, an object file is read whichdefines the shape of the object in terms of triangle vertices andtriangle surface characteristics.

At step 52 triangle strips are created by processing the data defined bythe object file. The strips are created in such a way that the hiddensurface problem is solved for all possible viewing directions, i.e. theobject file is made hidden surface removal tolerant (HSR tolerant), sothat when the strips have been created no care needs to be taken tosolve the hidden surface problem, independent of viewing direction andcamera position. Usually it is possible to process the complete objectfile in this manner whilst fulfilling the above criteria, however,sometimes it is impossible to create triangle strips in this manner forthe complete object because there is a conflict in the requiredrendering order between triangles of the object. In such cases theobject is divided into two new objects, which are each processedindependently like any other object.

In the next step, step 55, it is determined whether the objects have anyplanes of symmetry (this information is usually already available fromthe object file after the modeling procedure). All triangles at one sideof a plane of symmetry are removed and, if not present already, theplane of symmetry is stored in the object file.

The above steps may be required for a plurality of objects, and hence aquestion is asked at step 60 as to whether another object is to beprocessed. When answered in the affirmative, steps 50, 52, and 55 arerepeated, until all objects have been processed. All the steps 35 to 60are performed on a first device, typically a workstation with powerfulgraphics processing means, and once all object files have been processedthey can be transferred to a second device for display, a device withlesser graphics processing power, which is typically a mobile terminal,in the present embodiment a mobile phone 1.

Step 52 of FIG. 2, is shown in more detail in the flowchart in FIG. 4.In step 76, the vertex data of all the triangles 1-n of the object areread, starting with the number of vertices, the number of triangles, thevertex data and the triangle data. Based on the vertex data the triangleplanes for each of the triangles 1-n are calculated one after another instep 77.

In step 78 a new triangle strip is started by selecting a triangle thatdoes not occlude any of the unprocessed triangles. The selection of thefirst triangle may be done in a greedy style by simply choosing thefirst triangle in the set of unprocessed triangles.

According to an alternative embodiment of the invention, heuristicsearch techniques can be used to identify those triangles thatpotentially occlude the least number of remaining unprocessed triangles,e.g. by selecting triangles that do not have many unprocessed neighbors.Starting a new strip with thus identified triangles will result in fewerand longer triangles strips per object.

Several techniques are known to perform such a heuristic search andthese techniques are therefore not described in detail here.

The triangle selected to start the strip with is then compared with thetriangles in the set of unprocessed triangles to determine if theselected triangle can potentially occlude any of the unprocessedtriangles. A procedure for carrying out this determination is describedbelow with reference to FIG. 5.

In step 80 a neighboring triangle to be added to the triangle strip isselected. The selected triangle is then compared in step 81 with thetriangles in the set of unprocessed triangles to determine if theselected triangle can potentially occlude any of the unprocessedtriangles. The procedure for carrying out this determination isdescribed further below with reference to FIG. 5. The selected triangleis added to the strip in step 85 if the determination in step 81 isnegative. The selected triangle is then removed from the set ofunprocessed triangles in step 86. In step 87 the question is asked ifthe set with unprocessed triangles is empty. All triangles in the objectfile have been processed if the question asked in step 87 is answered inthe positive, and control is given to step 60 (FIG. 2). If the questionasked in step 87 is answered in the negative, control is given to step80 to select a new neighboring triangle to add to the triangle strip.

If the question asked in step 81 is answered in the positive, a checkfor a conflict is run in step 83, i.e. check if a circle exists such aswhen a first triangle can potentially occlude a second triangle, thesecond triangle can potentially occlude a third triangle and the thirdtriangle can potentially occlude the first triangle. Thus it isimpossible to create an object file in which the rendering order iscorrect for all possible viewing directions. If the check for a conflictis answered in the negative, a new triangle strip is started in step 78.If the check for a conflict is answered in the positive, the object isdivided using binary space partitioning techniques, e.g. by dividing theobject into two over the triangle surface of the last selected triangle.Control is then given to step 52 (FIG. 2), and the two “new” objects areprocessed like any of the other objects. The conflict check is performedby determining if any triangle can be selected such that it doesn'tpotentially cover any other unprocessed triangle. If this determinationis answered in the negative, the object needs to be divided.

As best shown in the flowchart of FIG. 5 a selected triangle can becompared with the unprocessed triangles starting with step 91, in whichthe value for a parameter X is set to 1 and the value for a parameter Yis set equal to the number of triangles in the set of unprocessedtriangles.

In step 92 the question is asked if the selected triangle has one ormore vertices in front of the plane of triangle X in the set ofunprocessed triangles. If the question asked in step 92 is answered inthe positive, the question is asked in step 93 if triangle X has one ormore vertices behind the plane of the selected triangle. If the questionasked in step 93 is also answered positive control is given to step 83(FIG. 4).

If the question asked in step 92 or 93 is answered in the negative, thevalue of parameter X is increased with one in step 94 and in step 95 thequestion is asked if the value of parameter X is equal to the value ofparameter Y. When the question asked in step 95 is answered in thenegative, the procedure of steps 92, 93, 94 and 95 is repeated until thequestion asked in step 95 is answered in the positive. Then control isgiven to step 85 to add the selected triangle to the triangle strip.

The above procedure is repeated until the set with unprocessed trianglesis empty, and the object file is in a format in which the triangles arein an order in which they can be displayed without needing to solve thehidden surface problem. Further, the triangles are arranged in trianglestrips and therefore the amount of calculations to display the object isfurther reduced.

The next step, step 55, which is not mandatory to the presentembodiment, is shown in more detail in the flowchart in FIG. 3. In step64 it is determined if the object is symmetrical about the XY plane.This data is usually available from the object file, in particular whenthe object file has been created using a graphics program or the like.In the unusual circumstance that the object file does not contain suchdata, e.g. because it reflects a scanned object, it would be possible toidentify planes of symmetry by assuming a plane of symmetry Z=0 and tocheck for every vertex with a positive z-value whether the same vertexexists with the same but negative z-value, and repeating the exercisefor planes of symmetry with Y=0 and X=0. Though not shown, it is alsopossible to include steps to rotate and translate the object so that anyaxis of symmetry align with one or more of the planes X=0, Y=0 or Z=0.

When symmetry about the XY plane is present, the plane of symmetry isstored in the object file in step 65, and all vertices with negativeZ-axis values are culled from the object file in step 66. Hereto, thevertex indices are reordered so that each mirrored vertex below thesymmetry plane comes after its pair above the symmetry plane, then theones above the symmetry plane are marked so that they have a mirroredcounterpart.

After step 65, or when no symmetry in the XY plane is detected, it isdetermined if the object is symmetrical in the XZ plane in step 67.

When symmetry in the XZ plane is present, the plane of symmetry isstored in the object file in step 68, and all vertices with negativeY-axis values are culled from the object file in step 69.

After step 69, or when no symmetry in the XZ plane is detected, it isdetermined if the object is symmetrical in the YZ plane in step 70.

When symmetry in the YZ plane is present, the plane of symmetry isstored in the object file in step 71, and all vertices with negativeX-axis values are culled from the object file in step 72.

After step 72, or when no symmetry in the YZ plane is detected, controlis moved to step 60, in FIG. 2. The amount of vertex data is the objectfile can thus be reduced by up to a factor of 8 depending on the numberof symmetry planes of the object. In practice, almost all objects usedin computer graphics have at least one and often two planes of symmetry,and thus a reduction in the amount of vertex data by a factor 2 to 4 isachievable. This is of significant importance when storing animated 3-Dgraphics on e.g. a mobile phone 1 as shown in FIG. 8 because of itsrelatively limited memory capacity.

When it is decided in step 60 that all objects 1-N have been thusprocessed, the object files are ready to be transferred from theworkstation to the mobile phone 1 for display.

The data transfer from the workstation to the mobile phone could be viaIR signals (IrDA) or RF signals, (Bluetooth) over the air as messages(SMS or MMS), cable, native (at production site) or any other suitablemedium.

As shown in FIG. 6, the object files 1-N are loaded onto the mobilephone in step 100 and stored on the random access memory (RAM) 17 of themobile phone 1 (cf. FIG. 8). The mobile phone is provided with aprocessor (CPU) 10 that is capable to write data to and read data fromthe RAM 17. The CPU 10 controls a digital signal processor 12 coupled toa microphone 14 and a speaker 15. The mobile phone 1 is further providedwith a subscriber identity module (SIM card) 16. An RFtransmitter—receiver circuit 18 is also connected to the CPU 10. Asoftware embedded 3-D engine 20 drives the LCD or plasma screen 30.

Since the amount of triangles in the object files 1-N has been reducedon the workstation in step 52, the amount of data is small enough to bestored in the relatively limited RAM 17 of the mobile phone 1. Theobject files have been prepared in the proper rendering order (HSRtolerant) and thus it is possible to enter the rendering stage as soonas the object order is determined. At step 101 (FIG. 6) input commandsare received which essentially define lighting and view parameters.Thereupon, the object rendering order is determined in step 102,preferably by using binary space partitioning techniques. The data ofthe first (most occluded) object to be rendered are read in step 103. Instep 104, the triangles that were removed from one side of a symmetrysurface are restored. The procedure is illustrated in more detail in theflowchart shown in FIG. 7.

In step 120 the question is asked whether a symmetry plane in the XYsurface is stored in the object file. If the question asked in step 120is answered in the affirmative, then a new set of vertices is created instep 122 by duplicating all vertex data presently in the object file andflipping the sign of the Z-axis values for the newly created vertices,so that the earlier removed vertices are restored, and the control ismoved to step 124. If the question asked in step 120 is answered in thenegative, the control is moved directly to step 124.

In step 124 the question is asked whether a symmetry plane in the XZsurface is stored in the object file. If the question asked in step 124is answered in the affirmative, a new set of vertices is created in step126 by duplicating all vertex data presently in the object file andflipping the sign of the Y-axis values for the newly created vertices,so that the earlier removed vertices are restored, and the control ismoved to step 128. If the question asked in step 124 is answered in thenegative, the control is moved directly to step 128.

In step 128 the question is asked whether a symmetry plane in the YZsurface is stored in the object file. If the question asked in step 128is answered in the affirmative, a new set of vertices is created in step130 by duplicating all vertex data presently in the object file andflipping the sign of the X-axis values for the newly created vertices,so that the earlier removed vertices are restored, and the control ismoved to step 105 (FIG. 6). If the question asked in step 128 isanswered in the negative, the control is moved directly to step 105(FIG. 6). Thus, all the vertices removed from the object file on theworkstation in step 52 are restored again.

In step 105 it is determined whether a given triangle's predefined frontface is visible with present viewing direction. Hereto, the direction ofthe front faces of the triangles 1-n in the object file are comparedwith the viewing direction, by calculating the dot product of the vectorof the viewing direction and the normal vector of the respectivetriangle. When this product is negative the triangle in question isculled since its front surface cannot be seen with the present viewingdirection. Thus, the amount of triangles is statistically reduced by50%.

Alternatively, a usually more accurate, often even faster (as it notnecessary to calculate normal, and the direction to camera changes withevery triangle as well) way of detecting backfacing triangles is tocalculate its area on the screen after projection. If the screen area isnegative, the triangle is backfacing.

In step 106 the transformation of vertices of the triangles from modelspace (i.e. the object file's reference coordinate system) into devicespace (i.e. the display or screen coordinate system) takes place using amatrix-vector multiplication. Methods and algorithms for vertextransformation are well known in the art and therefore not describedhere in detail.

When the vertices have been transformed in step 106 the vertex data issent to the display for viewing in step 107. In step 107 the trianglestrips are rendered in the order in which they were created, which inthis embodiment corresponds to the order in which they are stored in theobject file. The triangles are displayed in the order in which they wereadded to the respective triangle strip, which in this embodimentcorresponds to the order in which they are stored in the object file.Consequently, the triangles can be displayed in the order in which theyare stored in the object file without needing to address the hiddensurface problem whilst displaying the triangles.

At step 108 the question is asked if another object is to be displayed.If the question asked in step 108 is answered in the affirmative, thecontrol is moved to step 103 at which the data of the next object fileare read.

Thus, steps 103 to 107 are repeated until all objects have beendisplayed. When all objects have been displayed, the next frame is to bedisplayed and control is given to step 101 to receive the input commandsfor the next frame.

According to another embodiment of the invention, the mirrored verticesare removed as shown in the flowchart of FIG. 3, but the polygons arenot stripped, i.e. step 55 in FIG. 2 is omitted. Thus, a reduced sizeobject file is obtained that may be easily stored in the mobile phone,but when displaying the triangles it will be necessary to solve thehidden surface problem. Hereto, the painter's algorithm or theZ-buffering technique could be used on the mobile phone 1, the limits ofthe processing power of the mobile phone will though be reachedrelatively quickly.

The efficient techniques of the invention allow real time animated 3-Dgraphics on devices with relatively limited image processing hardware,such as mobile phones, PDAs and handheld video games. The invention istherefore not limited to the device described in the preferredembodiments (a mobile phone with a SIM card), the invention can beapplied any other type of mobile phone or device listed above.

Although the present invention has been described in detail for purposeof illustration, it is understood that such detail is solely for thatpurpose, and variations can be made therein by those skilled in the artwithout departing from the scope of the invention.

1. A method of preparing object files on a first device that are usedfor rendering two-dimensional images of three-dimensional objects on asecond device by processing object files containing data definingtriangle primitives representing at least one three-dimensional objectto determine a rendering order in which said triangle primitives may berendered for creating a two-dimensional image in which the hiddensurfaces are removed comprising the steps of: a) storing a plurality ofplanar triangle primitives corresponding to a plurality of surfaceportions of said at least one object in an object file, wherein saidplurality of triangle primitives are represented by vertex data arrangedaccording to an original order, b) selecting a triangle that does notpotentially occlude any of the unprocessed triangles to start a trianglestrip, c) selecting a neighboring triangle to add to the last trianglein the triangle strip, d) determining whether the selected neighboringtriangle has the potential to occlude any of the unprocessed triangles,e) adding the selected neighboring triangle to the strip and going backto step c) if the selected neighboring triangle does not have thepotential to occlude any of the unprocessed triangles, f) going back tostep b) to start a new triangle strip if the selected neighboringtriangle has the potential to occlude any of the unprocessed triangles,and g) repeating steps b) to f) until all the triangles in the objecttile are processed.
 2. A method according to claim 1, in which step b)comprises a heuristic search to identify those triangles for starting astrip that potentially allow long strips to be formed, preferably byselecting triangles that do not have many unprocessed neighbors.
 3. Amethod according to claim 1, further comprising the step of dividing theobject file into two, if at step b) it is impossible to find a trianglenot occluding other ones of the unprocessed triangles.
 4. A methodaccording to claim 3, in which the object is divided by using binaryspace partitioning techniques, preferably using any partitioning plane,so that at least one of the unprocessed triangles remains on a differentside than the unprocessed triangles it has the potential to occlude, inwhich a plane of an arbitrary unprocessed triangle is used and thetriangle is associated on the outward side of the splitting plane, thusleaving all the triangles that it may cover on the other side.
 5. Amethod according to claim 1, further comprising the step of determiningthe front side and the triangle planes of the triangles in the objectfile wherein the determination if a selected triangle can occlude any ofthe unprocessed triangles comprises the steps of determining whether: g)the selected triangle has one or more vertices on the front side of theplane of any of the unprocessed triangles, and h) whether any of theunprocessed triangles positively identified in step g) have one or morevertices on the backside of the plane of the selected triangle.
 6. Amethod according to claim 1, comprising further the following stepsbefore step b) i) determining if the object has symmetry in the XY-planeof the object space, j) storing a determined plane of symmetry in theobject file, and k) removing all vertices in the object file from oneside of a determined plane of symmetry, and repeating steps i) to k) forthe XZ-plane of the object space and for the YZ-plane of the objectspace.
 7. A method according to claim 1, further comprising the step of:l) transferring said object file or files from said first device to asecond device after step f) or k) and storing the object files on saidsecond device.
 8. A method according to claim 1, in which said firstdevice comprises a higher performance image processing means and saidsecond device comprises a lower performance image processing means.
 9. Amethod for displaying object files obtained by the method of claim 1, onsaid second device, comprising the steps of: m) determining the viewingdirection, and n) determining the object order, preferably using thepainter's algorithm and displaying said object or objects on said seconddevice.
 10. A method according to claim 9, further comprising the stepof: o) culling all triangles facing backwards relative to said viewingdirection.
 11. A method according to claim 9, further comprising thesteps of: p) retrieving planes of symmetry stored in the object file,and q) creating a duplication of the vertex data present in the objectfile, said duplication of the vertex data having the same order as theoriginal vertex data and being stored in the object file after theoriginal vertex data, and flipping the sign of the vertex values on theaxis orthogonal to the respective retrieved plane of symmetry of theduplicated vertices.
 12. A method according to claim 9, furthercomprising the steps of r) displaying the triangle strips in the sameorder as they were created, and s) displaying the triangles in thetriangle strips in the same order as they were added to respective onesof the triangle strips.
 13. A method according to claim 9, in which saidsecond device comprises a lower performance processing means.
 14. Amethod according to claim 13, in which the image rendering process iscarried out by the main CPU of the device, and all the further imagerendering means is software embedded.
 15. A method, according to claim9, in which said second device is a mobile phone.